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In  this  one-year  project  the  aim  was  to  continue  the  previous  three 
years'  research  on  designing  multimedia  instructions  for  procedures  ihci. 
people  could  use  and  learn  from  them  more  easily.  There  were  both  theoretica. 
and  practical  aspects  to  the  work.  We  viewed  the  conceptual  structure  of  a 
task  as  a  (hierarchical)  tree  (later  a  directed  acyclic  graph)  with  nodes 
representing  information  from  different  modalities  (motoric,  visual,  and 
linguistic).  >  This  theoretical  viewpoint  directed  the  experimental  work.  We 
will  give  beljow  an  overview  of  the  studies,  including  progress  and  results, 
and  sever^K  principles  for  instructional  design  which  came  from  the 
experiments. 

Whe  overview  will  be  preceded  by  a  short  summary,  stating  the  five  main 
studies,  the  total  number  of  subject  hours  run,  stimulus  materials  used,  and 
computer  programming  done.  At  the  end  are  listed  technical  reports, 
conference  papers,  and  publications. 


Main  Studies: 


(A)  Transforming  a  task's  tree  structure  to  lessen  short  term  memory  load  in 
instructions. 

(Bj  Developing  generic  and  functional  terminologyj 

(Cj  Theoretical  hypotheses  about  what  narration  in  dual  media  presentations 
should  consist  ofj 

(d)  Transfer  of  learning  in  assembly  tasks^  a^d 

(Ej  Empirical  investigation  of  adequacy  of  knowledge  representation  for  repair 
in  a  computerized  tutor. 

v  Number  of  subject  hours  run:  ,  T2DD 

S-Main  videotapes  used  as  stimulus  material;  The  Lift  (2  versions,  both 
prepared  by  Bud  Leonard,  a  television  director  from  Academic  Media  Services  at 
the  University  of  Colorado). 

Or  f 

Main  assembly  kits  used;  Fischer  Technik  50  and  Capsela  1000.  Also,  Eveready 
flashlights  were  used  for  repair  tasks. 

-'Supporting  programs*.  Finite  State  Grammar  for  Terminology,  M.  Kent,  1983; 
Computer  Techniques  roTxCluster  Analysis  (extended),  R.M.  Perry,  1984. 


Overview  of  Studies 


A.  Transforming  a  task's  tree  structure  to  lessen  short  term  memory  load  in 
instructions. 

In  previous  work  we  showed  that  videotape  instructions  based  on  a 
"majority"  conceptualization  of  an  object  (its  breakdown  into  subassemblies, 
subsubassemblies,  etc.)  yield  better  performance  than  those  based  on  a 
"minority"  conceptualization.  In  the  videotapes  used,  the  conceptualizations 
(trees)  were  traversed  in  a  top-down,  breadth-first  manner.  For  example,  the 
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entire  object  was  shown;  then  all  its  subassemblies  were  shown  and  assembled 
into  the  object;  then  each  subassembly's  subsubassemblies  were  shown  and 
assembled,  etc.  Our  first  major  question  in  this  research  was  whether,  using 
the  same  majority  conceptualization,  a  videotape  could  be  prepared  whose  shot 
sequencing  and  content  were  even  better. 

We  can  transform  mathematically  any  tree  T,  which  represents  a 
conceptualization,  into  a  conceptualization  T'.  T1,  when  traversed,  presents 

a  cued  sequence  of  actions.  (The  transformation  involves  editing  the  original 
concept  by  changing  the  relations  of  subconcepts.) 

The  key  difference  between  traversing  T  and  traversing  T‘  is  that,  in  T', 
anytime  a  new  subassembly  is  introduced,  it  is  directly  visually  preceded  by 
something  (a  piece  or  a  subassembly)  which  is  a  part  of  it.  This 
hypothetically  means  that  what  is  physically  present  immediately  before  cues 
the  concept  represented  by  the  assembly.  Another  difference  between  the 
videotapes  showing  T  and  T*  is  that,  in  T1,  two  cameras  are  used 
simultaneously.  One  camera  shows  (on  the  right  of  the  image)  the  goal  or 
subassembly  one  is  working  toward,  i.e.,  what  should  be  kept  in  mind.  The 
other  camera  shows  (on  the  left)  hands  getting  parts  and  putting  together  the 
subassembly.  In  T,  only  one  camera  was  used,  with  no  goal  shown. 

We  have  counted  *.he  numoer  of  concepts  which  the  central  processor  (in 
our  theoretical  framework)  must  keep  "on  hold"  during  traversal  of  T  versus 
T'.  We  can  show  that  resource  requirements  are  considerably  less  for  T'  than 
for  T. 

Two  groups  were  shown  videotapes,  based  on  T  and  T1  respectively, 
displaying  assembly  of  the  eighty-piece  object  (the  lift)  pictured  in  Figure 
1.  (The  names  given  to  subassemblies  are  shown  in  Figure  2.  Trees  T  and  T1 
for  the  lift  are  shown  in  Figure  3.)  Afterwards  subjects  built  the  lift  from 
memory.  Both  structural  and  functional  measures  were  made  on  their  memorial 
lifts.  For  structure,  the  number  of  correct  connections  was  counted.  For 
function,  whether  or  not  the  lift  worked  was  assessed.  Table  1  gives  the 
results. 

A  2  x  2  (T1  vs  T  videotape;  male  vs  female)  between  subjects  ANOVA  on  the 
number  of  correct  connections  showed  no  main  effect  of  either  videotape  or 
gender,  but  an  interaction  of  videotape  x  gender.  Females  (novices  in  their 
own  estimation)  perform  significantly  better  when  instructed  by  T’,  and  for 
the  first  time  in  any  assembly  task  in  this  research,  their  performance  equals 
that  of  males. 

The  experimental  result  yields  a  very  precise  and  easy-to-follow 
guideline  for  visual  presentation  of  instructional  material  which 
significantly  helps  novices. 


B.  Developing  generic  and  functional  terminology. 

A  second  major  question,  again  derived  from  our  earlier  work,  was  how 
functionality  (the  way  pieces  in  an  assembly  kit  are  used)  influences  the 
names  a  person  gives  to  the  pieces.  We  previously  developed  a  methodological 
schema  for  getting  good  names  for  parts.  Basically,  the  names  were  generated 
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by  people  and  the  "majority"  names  were  chosen.  They  could  be  iteratively 
improved,  based  on  matching  and  recall.  We  used  a  part  of  the  same  method  in 
this  new  research. 

Here  we  were  concerned  with  two  kinds  of  names:  generic  vs  functional. 
An  example  of  a  generic  name  is  1/2  x  S  x  8"  cedar  board.  Examples  of 
functional  names  are  shingle,  cutting  board,  lid  for  box,  and  piece  of 
firewood.  Often  generic  and  functional  names  will  be  identical,  typically  if 
a  given  object  has  only  one  use,  for  example,  a  hammer. 

We  needed  an  operational  definition  to  decide  if  a  name  was  generic  or 
functional.  The  experimental  schema  was  easy: 

(1)  Show  the  subject  an  object  "out  of  context."  The  subject  generates  a 
name  for  it.  This  is  a  generic  name. 

(2)  The  subject  uses  the  object  for  some  specific  purpose.  While  doing 
so,  the  subject  generates  a  name  for  it.  This  is  a  functional  name. 

In  the  experiment,  we  were  interested  in  the  transfer  of  the  functional 
name,  generated  at  the  time  of  use,  into  the  generic  name,  generated  later, 
out  of  context.  We  had  subjects  use  an  object  for  a  specific  purpose.  Then 
later,  "out  of  context,"  subjects  were  asked  to  name  it.  Names  generated  by 
such  subjects  could  be  compared  with  names  by  people  who  never  used  the  object 
for  that  purpose,  to  measure  how  frequently  or  strongly  functional  names 
become  (are  generated  as)  generic  names.  There  were  five  groups  of  subjects: 


(1)  (2) 

Name  46  Build  objects 

pieces  of  A1  and  A2. 

Capsela  kit  Name  46  pieces 
out  of  out  of 

context.  context. 


The  analysis  was  done  using  software  written  by  Marty  Kent  (1983).  It 
selects  the  most  commonly  generated  names.  There  was  a  strong  shift  from 
functional  to  generic  names,  i.e.,  people  used  functional  names  as  generic 
names.  For  26%  of  the  names  (12  of  46)  the  majority  (or  plurality)  name  was 
di fferent  for  group  1  versus  groups  2,  3,  4,  and  5.  Table  2  gives  some 

examples. 

There  was  evidence  of  lateral  transfer.  Within  a  given  group,  functional 
names  were  used  for  objects  simi 1 ar  to  ones  used  in  the  actual  construction. 
For  example,  group  3  never  used  piece  #13  as  a  propeller,  but  they  did  use  piece 
#12  as  a  propeller.  (See  Table  2.) 

Many  names  (plurality)  did  not  change  at  all.  In  many  situations,  this  was 
expected,  because,  if  in  the  task  the  functional  name  coincided  with  the 
generic,  the  subject  would  not  switch  the  name. 


(3)  (4)  (b) 

Build  objects  Build  A1  Build  Bl,  B2, 

Bl,  B2,  and  and  A2.  and  B3 

B3. _  Wait  48  hours.  Wait  48  hours. 

Name  46  pieces  Build  Bl,  B2  Build  A1  and  A2. 

out  of  context.  and  B3.  Name  46  pieces 

Name  46  pieces  out  of  context, 
out  of  context. 
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When  subjects  used  a  piece  for  two  different  purposes,  separated  by  a  48- 
hour  delay,  the  most  recent  purpose  determined  the  name. 

We  think  that  these  naming  results  are  interesting  not  just  as  a  linguistic 
observation,  but  because  they  have  some  predictive  value.  The  names  subjects 
use  indicate  how  they  conceptualize  an  object.  We  could  expect  negative 
transfer  of  learning  when  subjects  are  given  a  task  in  which  objects  have  to  be 
used  functionally  in  different  ways.  For  example,  suppose  an  object's  name 
(because  of  use)  becomes  "float,"  and  now  it  is  used  as  a  propeller.  We  expect 
that  subjects  would  make  errors,  or  not  spot  errors  of  use. 


Our  general  hypothesis  for  part  names,  for  good  transfer,  is:  construct 
names  based  on  generic  concepts,  and  extend  them  for  specific  uses.  Here  is  an 
example  of  a  structure  of  a  concept  we  want  subjects  to  build: 


Object: 


X 


functionality  1 
Use  the  yellow  ball 
as  a  float. 


functionality  2 
Use  the  yellow  ball 
as  a  head. 


functionality  3 
Use  the  yellow  bal 1 
as  a  robot  arm. 


\ 


C.  Theoretical  hypotheses  about  what  the  narration  should  consist  of  in  dual 
madia  assembly  instructions. 

Below  is  a  list  of  observations  and  suggestions  derived  from  our 
theoretical  framework.  They  were  used  in  designing  the  narration  for  our  T' 
assembly  videotape. 

1.  Instructions  supposedly  build  concepts,  the  relation  of  subconcepts, 
and  the  relation  of  links.  They  also  build  the  identity  relation:  when  an 
object  is  shown  a  second  time,  it  needs  to  be  identified  as  the  same  as  before. 
(See  4  below.) 

2.  Narration  is  secondary  to  visuals.  This  means  the  narration  should  not 
form  an  independent  text.  Thus,  as  little  grammar  as  possible  should  be  used. 
The  narration  should  be  limited  to  the  concepts  visually  presented,  with  no 
extraneous  material. 

3.  The  role  of  narration  should  be  to  direct  attention  for  building  the 
concept.  Narration  should  not  precede,  but  be  Tn  synchrony  with  or  later  than, 
the  visuals  (Baggett,  1984). 

4.  In  establishing  the  identity  relation,  only  names  for  subassemblies 
that  have  been  shown  visually  should  be  used.  When  there  is  a  visual 
orientation  change,  establishing  identity  verbally  is  especially  important. 
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5.  Early  narration  should  be  limited.  Narration  should  become  richer  as 
the  end  is  approached. 

6.  Via  motion,  the  motoric  component  is  visually  shown.  The  proper  verbal 
comment  is  to  describe  the  action  (using  the  imperative)  and  then  the  result,  to 
parallel  the  visual:  "Press  the  wheel  with  your  thumb  to  snap  it  on  the  axle." 

7.  Numbers  should  be  used  freely.  This  helps  form  the  concept  of  a  group 
of  pieces. 

D.  Transfer  of  learning  in  assembly  tasks. 

The  fourth  area  of  study  was  transfer  of  learning.  The  basic  question  was 
whether,  when  a  person  builds  two  objects  A  and  B,  with  a  common  substructure  C, 
the  conceptualization  evidenced  in  C  as  a  part  of  A  (built  first)  will  also  be 
present  as  a  a  part  of  B  (built  second).  If  so,  we  will  say  there  has  been 
transfer  of  learning  (transfer  of  conceptualization).  (In  our  earlier  work,  we 
gave  a  method  to  derive  a  person's  hypothetical  conceptualization  of  an  object 
he  is  building,  based  on  the  order  in  which  the  person  uses  the  pieces  in 
bui 1  ding. ) 

There  were  3  groups  of  subjects  each  containing  12  males  and  12  females: 
Group  Task 

1  Build  C  only  (a  substructure  common  to  A  and  B) 

2  Build  A  first;  build  B  second. 

3  Build  B  first;  build  A  second. 

"A"  was  a  model  helicopter  with  54  pieces  and  58  connections;  "B"  was  an 
eggbeater  with  60  pieces  and  70  connections.  Both  were  made  from  the  Fischer- 
Technik  50  assembly  kit. 

We  did  the  following  data  analysis.  In  all  groups,  we  focussed  on  the 
conceptualization  of  C.  There  were  three  analyses: 

(1)  Look  at  conceptualization  of  the  whole  object  (helicopter  or  eggbeater),  and 
then  restrict  it  to  C. 


(2)  From  the  original  order  of  selection,  we  can  strike  out  all  pieces  not 
belonging  to  C,  and  find  the  conceptualization  of  C  alone,  in  one  of  2  ways: 


(a)  renumber  the  selections  in  C,  so  that  they  range  from  1  to  24,  the 
number  of  pieces  in  C. 


(b)  do  not  renumber  the  selections  in  C,  so  that  the  range  is  greater  than  1 
to  24. 

At  the  close  of  this  project,  the  analysis,  done  with  software  written  by 
R.M.  Perry,  was  in  progress.  We  report  a  preliminary  analysis,  found  after  the 
end  of  the  project.  We  will  say  more  transfer  of  learning  has  occurred  in 
situation  X  than  in  situation  Y  if  the  distance  between  conceptualizations  built 
in  X  is  less  than  the  distance  between  conceptualizations  in  Y. 
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Method  of 

Average  distance  from 

Average  distance  from 

deriving 

helicopter  built  first 

eggbeater  built  first  to 

conceptual ization 

to  eggbeater  built  second 
(HI  -  E2) 

helicopter  built  second 
(El  -  H2) 

(1) 

56.4 

59.2 

(2a) 

55.9 

62.8 

(2b) 

61.5 

68.1 

The  helicopter  (with  54 

pieces  and  58  connections) 

can  be  viewed  as  a  simpler 

object  than  the  eggbeater  (60  pieces  and  70  connections).  It  appears  that  the 
conceptualizations  from  HI  and  E2  are  somewhat  closer  together  than  those  from 
El  to  H2.  (No  statistics  have  been  performed.)  A  tentative  conclusion  is  that 
there  is  more  transfer  (as  measured  by  similarity  of  structure)  from  a  simple  to 
a  complex  object  than  from  a  complex  to  a  simple  one.  If  the  result  holds  up 
with  further  analysis,  it  is  easy  to  interpret  theoretically.  Presumably  a  more 
complex  object  has  a  more  complex  (deeper)  structure.  If  it  is  practiced  first 
and  a  part  of  the  complex  structure  occurs  in  a  simpler  structure  later  (the 
case  of  E1-H2),  a  person  may  not  be  able  to  locate  the  part  in  the  earlier 
structure  because  it  is  so  deep  in  the  hierarchy.  Thus,  very  little  transfer. 
In  the  case  of  simple  first,  complex  second  (H1-E2),  more  transfer  is  evident 
because  the  person  has  less  difficulty  locating  the  common  part  in  the  simple 
(shallower)  structure  and  using  it  later.  These  comments  are  tentative;  further 
data  analysis  is  needed. 


E.  Empirical  investigation  of  adequacy  of  knowledge  representation  for  repair 
in  a  computerized  tutor. 

One  goal  of  this  project  was  to  specify  what  should  be  the  content  of 
interactive  instructions  for  repair.  A  subgoal  was  to  specify  what  knowledge 
representation(s)  of  the  object  to  be  repaired  should  be  in  the  program.  We 
investigated  what  knowledge  to  include  by  collecting  repair  data  as  follows. 

Subjects  were  asked  to  repair  two  malfunctioning  objects,  a  flashlight  and 
an  unusual  object  made  from  the  Capsela  1U00  kit.  Each  subject  had  available  a 
helper  who  had  some  knowledge  which  might  aid  in  doing  the  repair.  In  the  case 
of  the  flashlight,  the  knowledge  came  from  the  fictitious  adult 
conceptualization  of  the  flashlight  given  in  the  addendum  to  the  proposal 
Principles  of  Instruction  for  Successful  Maintenance  and  Repair.  In  the  case  of 
the  second  object,  a  similar  conceptualization,  consisting  of  nodes  representing 
actions,  visual  concepts,  names,  and  abstract  concepts,  was  created  and  used. 
Using  only  the  knowledge  represented  in  these  conceptualizations,  the  helper 
answered  the  subject's  questions,  either  verbally  or  by  demonstrating  actions 
which  were  within  the  helper's  conceptualization.  If  the  subject  requested 
information  not  in  the  representation,  the  helper  responded,  "I  don't  know." 

The  subject  questioned  the  helper  and  performed  actions  on  the  objects 
until  they  were  repaired.  The  helper  recorded  the  actions  and  requests  of  the 
subject.  The  sessions  were  tape-recorded;  a  few  were  videotaped. 

At  the  Tucson  conference  (1984)  we  presented  the  hypothetical  multimedia 
conceptualization  of  a  flashlight  mentioned  above.  It  is  the  specific 
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conceptualization  that  our  expert  helper  pretended  to  have,  in  our  repair 
experiment.  We  showed  how  to  interpret  a  subject's  actions  while  performing  a 
repair  according  to  this  written-down  conceptualization: 

The  actions  of  the  subject  as  observed  by  the  experimenter  must  be  matched 
with  the  nodes  of  the  graph  as  written.  This  is  a  direct  use  of  the 
representation.  The  result  is  a  traversal  of  the  nodes  of  the  concept.  Using 
the  performance  of  a  specific  subject  doing  a  repair,  we  traced  her  conceptual 
walk  on  our  graph.  To  get  the  subject's  hypothetical  partial  conceptualization, 
we  take  all  nodes  visited  by  the  subject  and  close  them  upward  by  the  subconcept 
relation,  to  make  a  complete  graph.  The  more  tasks  the  subject  performs,  the 
better  the  graph  we  get. 

One  can  iterate  this  process,  comparing  the  graph  of  the  helper  and  the 
subject,  and  adding  new  subjects,  until  nothing  has  to  be  added  to  the  helper's 
concept.  One  can  then  drop  from  the  conceptualization  all  nodes  that  are  never 
used.  The  resulting  graph  can  be  used  as  a  basic  data  structure  for  a 
computerized  helper.  (Some  modifications  of  the  graph  could  be  done  at  run 
time,  as  we  showed  in  the  Tucson  talk.) 

The  knowledge  representation  is  also  a  tool  for  the  following  problem:  The 
experimenter  observes  the  subject  and  tries  to  analyze  what  went  on.  The  final 
product  of  the  analysis  is  often  confused.  Here,  we  start  with  our  (objective) 
conceptualization.  When  we  score  the  subject,  all  we  have  to  do  is  mark  nodes 
on  a  graph.  So  we  see  it  as  a  workable  way  to  do  complex  data  analysis  for 
complex  situations,  including  motoric  ones  and  interactions  with  the 
experimenter. 

Finally,  in  experimental  work,  we  examined  an  actual  repair  situation  (on 
an  object  from  the  Capsela  kit)  and  showed  how  some  aspects  of  our  planned 
program/subject  interactions  would  be  carried  out.  We  were  even  able  to  show 
that,  with  a  properly  prepared  graph  structure,  the  program  will  outperform  an 
expert  helper  in  giving  advice. 

Final  Remarks: 

We  summarize  several  findings  from  the  project: 

1.  We  have  an  easy-to-follow  blueprint  for  the  visual  presentation  of 
instructional  material.  Novices  perform  as  well  as  more  experienced  people 
when  given  instructions  so  designed. 

2.  We  suggest  a  multiple  naming  schema  for  unfamiliar  parts  which  have  two  or 
more  functions.  Names  should  be  constructed  based  on  generic  concepts,  and 
extended  for  specific  uses. 

3.  We  have  given  -.even  guidelines  for  -designing  narration  for  video  and  film 
instruction. 

4.  For  good  transfer  of  learning,  tasks  should  be  ordered  with  simple  ones 
early  and  more  complex  ones  later. 

5.  We  have  a  multimedia  knowledge  representation  whose  implementation  looks 
promising  as  a  data  structure  for  an  intelligent  tutor  to  help  with  assembly 
and  repair  tasks. 
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Table  1 

Number  of  correct  connections  and  proportion  functional  in  assembly  from 
memory.  Table  a  presents  results  after  viewing  the  tree-transformed  videotape. 


T'.  Table  b 

contains  results 

after  viewing 

videotape  T. 

average  number 
of  correct 
connections 

proportion 

functional 

average  time 
to  work 
(in  minutes) 

average 
number  of 
pieces  used 

number  of 

engineering 

majors 

mal  es 

(16  subjects) 

49.4 

9/16 

70.8 

76.8 

1 

females 
(16  subjects) 

51.9 

8/16 

59.1 

70.1 

0 

a.  "tree-transformed"  videotape,  T' 


males 

(16  subjects) 

52.6 

15/16 

59.4 

78.6 

3 

females 
(13  subjects) 

37.2 

4/13 

64.6 

65.5 

1 

b.  videotape  T 


O  v* 
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Table  2.  Most  commonly  generated  name  for  pieces. 


piece 

number  1 


•’f&p 

wrench 


13 


rx' 

'*+■ 


wrench 


most  commonly  generated  name 


2 

GROUP 

3 

4 

smal  1 

propel ler 

propel ler 

connector 

large 

propel  1 er 

propel ler 

connector 

5 

small  bar 

propel  1 er 


T- 

Platf orra 


String 

Guide 


Pulley  and  String 


Carrier  Roller 

base  wheel 

Assembl 


Figure  2 


